A Mixed Methods Approach to Evaluation of Large Scale Participatory Processes

Evaluation is critical to understanding how participatory processes work and how they can be structured to maximize the benefits of stakeholder and decision maker collaboration. This presentation provides a framework for evaluating stakeholder involvement in decision making in science and technology. Specifically, the following topics are covered: 1) the importance of incorporating program evaluation in processes that involve participation in decision making; 2) a review of models for evaluating participatory processes; 3) a new model for evaluating stakeholder involvement that incorporates a mixed method design; and 4) the advantages of a mixed model evaluation approach to inform the field about the types of participatory processes that work best for particular stakeholders, issues, and desired outcomes. The mixed methods model was used to evaluate the Public Engagement Pilot Project for Pandemic Influenza, an effort to involve citizens and stakeholders in policy decisions regarding vaccine distribution in the United States. Using this example, the authors describe how the mixed methods framework was used to jointly identify the evaluation questions with stakeholders and partners, select qualitative and quantitative measures, determine appropriate analyses, and effectively communicate results to decision makers and stakeholders. The implications for evaluating other public and stakeholder engagement processes are discussed.